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FOREWORD 


Evaluation is an. idea whose time has arrived. School systems 
can and are presenting the public with much information on how pupils 
are faring in various cognitive areas. This pamphlet offers some 
excellent common sense approaches to evaluation which all too often are 
neglected. 

Assessment of an educational system can be done without using 
elaborate statistical techniques. Some very simple ways of analyzing 
data and presenting information are quite useful. In fact the layman 
can better understand statistics presented in chart form than those 
couched in statistical jargon. 

This manual springs from concerns expressed by school personnel 
to Gerald H. Wohlferd, its author, and to Charles M. Armstrong, now 
retired. Accordingly, the manual is presented for the use of those 
school personnel who feel a desire to assess the quality of their 
educational system, but who would welcome suggestions as to how the assess¬ 
ment might be done. It is designed to fill the gap between no analysis 
and refined statistical analysis. The illustrated analytical methods 
presented herein are elementary, and basic data needs are readily 
satisfied. Those interested in using any of the procedures outlined 
in the text may contact the Bureau of School Programs Evaluation for 
needed help. 


for 
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SIMPLIFIED EDUCATIONAL ASSESSMENT 


Introduction 

How good is my school? This question, which at first glance appears 
to be sensible and straightforward, is in fact very difficult to answer. No 
single answer can be given since many courses of study, grade levels, rooms 
and diverse individuals compose a school. The question, then, is really a 
series of questions whose answers, when taken together, give an indication 
of the quality of education being provided. Some of the questions whose 
answer might better be sought are: How do the children in my school com¬ 
pare with other children in the nation, the state, among themselves? How 
have the children progressed over the years? Are the teachers equally 
effective in teaching? 

Some of the above questions are being answered every day. Others 
are avoided because of clerical costs involved in collecting and preparing 
data for analysis, or because of ignorance of statistics. Fortunately, 
simple analyses are not costly, nor is a comprehensive knowledge of statis¬ 
tics necessary. Too, most of the data, necessary to a simple analysis, are 
already on hand or are easily obtained. Often all that is needed is a 
reordering, classifying, or sorting of available data. 

Asking the Right Question 

However, before educational data analysis is started a few helpful 
hints might be in order. The first is to ask only those questions which 
can be answered. For example, the question, "How do the children in my 
school compare with other children in the nation?," is quickly recognized 
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as being too broad. In its present form it can not be answered. One would 
want to specify at least as to subject area and grade level. The question 
would better be phrased^ "How do the third grade children in ray school 
compare in reading skills with other children in the nation?" If desired, 
further specifications can be made, so that the subject area is delineated 
by the particular reading skill involved, and the type of student is 
identified. The original question would thus be supplanted by multiple 
questions, such as: "How do the third grade boys in my school compare 
in reading comprehension skills with other third grade boys in the nation?" 
and, "How do the third grade boys in my school compare in reading vocabu¬ 
lary skills with other third grade boys across the nation?", etc. 

Another hint in making school evaluations is to ask questions which 
are of value--questions which are usable as a basis for management and 
curriculum decisions. For example, to ask how many kindergarten students 
can multiply fractions would be a waste of time. One might better ask about 
the reading vocabulary skills of boys and girls in order to decide whether 
more emphasis should be placed upon skill building in that subject area in 
planning subsequent curriculum activities. 

One must expect that answers to evaluation questions may reveal 
shortcomings as well strengths. The strengths may then be capitalized 
upon, while the shortcomings may be used as points of discussion of future 
administrative policy, curricula, organization, or program changes. 

A third hint is that evaluation is like quicksand in that the 
answer to one question leads to asking further questions and into even 
deeper data analyses. Sooner or later a point is reached where a district 
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must decide that it has gone far enough. Therefore, it might be xi7ise to 
set up a tentative evaluation program at the start, which establishes the 
limits to the evaluation. 

Accuracy of Data 

A caution about the accuracy of data is in order. No single piece 
of datum should be accepted as an accurate measure until it is at least 
checked for sensibleness, and possibly duplicated at another time. Thus, 
a single I.Q. measure on a child must only be considered accurate within 
a range of fifteen points on either side of the obtained score. A second 
score which is about the same would lend some assurance that both scores 
are probably right. 

In the same vein, be increasingly more suspicious of data as it 
passes through several people before being used in any analysis. Each 
time figures are transferred from one source to another the chance of error 
is increased. No one is infallible. Most of us have dialed a wrong tele¬ 
phone number at some time. Nor is human error the sole source of poor data. 
Computers do make errors. Usually they produce such whoppers that the 
errors are quickly seen. But dirty equipment, old or cheap tapes, brittle 
cards, or temperature variations, can produce difficult-to-discover errors. 
Therefore, be suspicious of your data, whether it is hand copied or computer 
compiled, and cautious in making judgments from it. 

Finally, an answer to a question needs to be supported by subsequent 
analysis before drastic changes are made in school policy or program. One 
research finding is a hint; the second similar finding is a suggestion; the 
third is probable proof or establishes a trend. 
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Comparison to National Norms 

The following sections will show how various educational questions 
can be answered without recourse to detailed statistical analyses or 
expensive data handling. Most school districts conduct an achievement 
testing program on a yearly schedule. The purpose of such a testing pro¬ 
gram can be twofold. First, the tests may have pupil diagnostic capabil¬ 
ities. The question answered in this situation is "On what specific 
knowledge segments do individual children require remedial or catch-up 
help?" Since the question answered is not, in this form, a school evalu¬ 
ation question, it will not be discussed further at this time. Second, 
the test results are usually stated either in terms of averages or per¬ 
centiles. The question answered in this situation is, "How do my children 
(by grades in which tests are given) compare in achievement (by subject ■ 
and subtests available) with other children in the nation?" 

Comparison Over Time 

The latter analysis, i.e., a comparison to national norms, is known 
as a status analysis, because only one point in time is considered. As a 
status analysis, it has limited value since children in the school district 
seldom match the national sample on which the test norm tables are based. 
The real value is in comparison of district scores over time. As long as 
the test battery remains the same, and the norm tables which are matched to 
it remain the same, the test norms provide a stable scale against which to 
compare one yearns achievement with scores of preceding years. District 
averages reported as grade equivalents or percentiles may be plotted on a 
chart such as Figure 1. More than one test area, such as related subtest 
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scores, may be recorded at the same time. Many simultaneous plottings 
may, however, lead to confusion rather than clarity. 
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Figure 1. Third Grade Achievement Over Time 


Figure 1 would indicate to the district (hypothetical) for which third 
grade reading and arithmetic grade equivalent scores were plotted, that 
their students are achieving near the national average (3.0) as established 
by the test company. Between 1969 and 1971 arithmetic averages had an 
upward trend. Subsequently they leveled off. Reading slumped for a number 
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o£ years and then spurted upward. The administrator would want to look for 
reasons why the grade equivalents had changed. Were the gains in arith" 
metic and losses in reading due to a change of scheduling in 1969 which 
assigned fifteen minutes of dally second grade reading time to arithmetic? 
Could the gain be attributed to a successful Title I compensatory education 
program? Was the gain in reading possibly due to a new reading curriculum 
established in 1972? If the latter, was it because the children actually 
read better, or was the curriculum now oriented toward teaching items or 
skills tested in the reading test? Will the gain in reading scores be 
maintained? 

These and other questions will have to be answered through further 
study and experimentation. The change of arithmetic scores may not have 
been due to a scheduling change. Experimentation with an altered schedule 
in a school building, or in a single room if the district is small, could 
possibly give an answer to why the arithmetic and reading scores changed. 
Only time would tell If the last year's reading gain would be maintained. 
However, study of test items in comparison with both old and new curricular 
materials may give a tentative reason for the sudden rise in reading scores 
even before next year's test results are secured. 

As can be seen from the above discussion, the use of simple statis¬ 
tical methods can lead to more questions. When a change in score occurs, 
the obvious response is to ask, "Why did the scores change?" A sensible 
reason for the change should be searched for. Not always are changes due to 
school-controlled situations. The real reason for the change in reading 
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scores in the illustration above could have been something external from the 
school, such as the starting of a summer reading program at the local library, 
an influx of able students from a local private or parochial school which 
closed, or the cumulative effect of a TV program such as Sesame Street. Thus, 
the school administrator should look to areas both under and outside the con¬ 
trol of the school for possible causes of change of achievement scores. 

Subclassification 

One method of searching for out-of-school causes is to divide the 
children into various subgroups and compare the achievement averages of 
the groups. One common division is by length of time in attendance in 
the school district. New entrants can depress or raise achievement 
averages. New homes can add a different type of student depending upon 
the cost of the housing. The question then evolves to, "What type of 
student is causing the scores to change?" Or, "Do different student 
subgroups achieve at differing levels?" 

Should subgroups be found to achieve at different levels, special 
programs can be designed to help alleviate those with depressed scores. 

For example, a district which covers a wide range of community types may 
wish to group the records of the children by community type, such as urban 
central, suburban, and rural fringe. Should the early elementary rural 
fringe students have lower reading scores, a search might be made for story¬ 
books more closely related to rural living. Should the urban central stu¬ 
dents show retarded learning skills, the administration may wish to place 
more emphasis upon parental participation on PTA and other school affairs. 
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Attentlon directed to the educational and social needs of specific sub¬ 
groups X’jlll, if successful, have the effect of raising the total group 
average. The above discussion points out that total group averages can 
hide the differing academic levels of subgroups. 

New York State, because of its mandated statewide testing, is one of 
the few states in which districts can answer the question, "How do the chil¬ 
dren in my school district compare with other children in the state?" Of 
course, the question has to be rephrased so it is more specific as to cur¬ 
ricular area and grade level. The state "Pupil Evaluation Program, School 
Administrators Manual" August, 1974, offers many valuable suggestions for 
analyzing the data provided in the yearly report that help answer the above 
question. One addition to their suggestions could be made. On page 15 a 
chart is shown which compares over a number of years for two school buildings 
the percent of students achieving below the Statewide reference point, A 
school district can change the chart by omitting one school building, instead 
substituting the percent of students in the average category, and the per¬ 
cent of students in the above average category for the remaining school 
(see page 11 of PEP School Administrators Manual for method of securing 
percents). 

Figure 2 Illustrates how percents for students grouped by ability 
may be plotted together. It shows what can happen when attention is 
focused upon a particular group of students. In 1968 and 1969 the district 
was operating so that each group of students was achieving at about the same 
level in the subject tested. The district then became concerned with the 
increase in the percent of students in the "below average" category. 

Teachers were urged to place greater emphasis upon reducing the percent of 
students in that category. The program was successful in 1971 and 1972. 
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However, there was a simultaneous drop in the percent of students from the 
"above average" groupo The trend continued in 1972. Thereafter, greater 
attention was given to the loss of students from the "above average" group. 
Subsequently, the percent of pupils in the "above average" group rose. 
However, so did the percent of students in the "below average" group. 



Figure 2. Percent of Total Reading Scores Over Years 
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The above analysis illustrates the need of asking an important 
question whenever a change of policy or program has been placed into 
operation. It is, "What are the results, positive or negative, which occur 
other than those expected?" Attention solely to the "below average" 
category would have missed the effect upon the "above average" group, 
unless proper data analysis was conducted. 

Standard Deviation 

Commercial test companies also report scores in terms of standard 
deviations. A standard deviation is a calculation of the general spread 
of scores. It shows how closely the scores are grouped around the average 
(or mean). About 68 percent of the scores fall within the range of one 
standard deviation above the mean to one standard deviation below the mean 
The standard deviation changes as the spread of student scores changes, 
becoming greater as the spread of scores increases and becoming less when 
scores group more closely to the mean. Thus, comparison of standard 
deviations over the years, roughly answers the question, "Are the students 
in a grade in succeeding years retaining the same closeness or spread of 
scores? This question, may be crucial when a new program or theory of 
learning is tried. For example, Figure 3 illustrates how the standard 
deviations can be added to and subtracted from the averages and then 
plotted. The increase in the spread of scores starting in 1969 might be 
the result of greater emphasis on allowing each child to progress at his 


own speed. 
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Figure 3. Spread of Scores Around Average Over Time 


The reduction in spread of scores between 1972 and 1973 could have resulted 
from a policy to give special help to the slower students. Bringing their 
scores closer to the average would have the effect of reducing the standard 
deviation. 


Distributions of Scores 

As illustrated in some of the text, the question, "How are my 
children achieving?" can sometimes be better answered if subgroups are used 
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to form new averages for comparison. Subgroups were chosen in those illus¬ 
trations on the basis of theory or experience. Some natural subclassifi¬ 
cations come readily to mind. Sex, age, achievement level, home background, 
and type of community surroundings are but a few that can be used. There 
are times when the usual classification schemes do not yield insights into 
how adequately students are achieving and why. Since all programs are not 
equally successful for all pupils, the question might then be asked, "Wlrich 
students are benefiting greatly from school and which students might be 
affected adversely?" The reason for asking this question springs from the 
theory that a good school is one that constantly tries to make it easier 
for its children to learn. Those program aspects which are effective can 
be incorporated into other learning situations. Those ineffectual should 
be discontinued. 

Frequency Distributions 

Often, identification of helpful or harmful aspects are hard to 
discover using the preceding data analysis methods, A rewarding method of 
looking at pupil scores is to chart them as distributions. Distributions 

of student scores which can reveal differing effects to be taking place, 
are constructed from listings of student scores, as in Table 1, From 
the list of student scores, the number of scores by group is determined. 

A chart of the number (frequency) of students for each score is then 
constructed. Figure 4 is an example of a frequency diagram (in this case it 
is a line graph) constructed from the listing of scores in Table 1. 
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Table 1 

List of Student Scores and Numbers in Groups 
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The frequency of scores should ideally assume the rough shape of 
a bell. Gross irregularities from a bell shape is an indication that 
some force may be in operation which causes scores to move to new values. 



The distribution of scores in Figure 4 is biomodal, that is, there are 
two humps. Special attention should obviously be focused upon the chil¬ 
dren who compose the lower hump, and reasons for their low grouping should 
be investigated. In this example the test scores could have represented a 
situation where three classrooms were combined for an audiovisual presen— 
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tation of the course content. The students met in the cafeteria at one end 
of which a television set was placed. When students in the lower (mode) 
were identified by name and ability, there did not at first seem to be any 
sensible reason for their low scores. A teacher in charge of the combined 
class finally saw a pattern. A large proportion of the lower mode was 
composed of students who sat in the back of the room. It was then quickly 
determined that these children were too far from the television set for 
adequate viewing. Also, noise coming from the kitchen made hearing difficult. 
Furthermore, upon further analysis of scores in the lower hump it was found 
that a few pupils were negatively affected because the surroundings were 
quite dissimilar from the traditional classroom setting. However, due to 
the general success of the audiovisual presentation, a decision was made 
to purchase more television sets, and to move the children back into their 
three regular classrooms where the T.V. programs could be seen and heard by 
all children. 

The foregoing discussion has pointed out how distributions of student 
scores can be used to determine probable causes of poor education. Questions 
which might be answered are; "Are the educational programs of the school 
equally effective for all children?", "If not, which children are being 
adversely affected?", "Wliy were these children adversely affected?", and 
"What changes can be made in order that adverse affects can be removed?" 

Scattergrams 

Several ways of searching for answers to the question, "How good 
is my school?", have been presented. Previous discussions have used the 
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average as a basis of comparison, even though children of different 
abilities compose the group for whom the average has been found. Though 
the previous analytical techniques can offer means of gaining insights 
into school quality, a most important question has been omitted. It is, 

"Are the children progressing at a proper speed?" Fundamental to this 
question is the theory that a good school is also a school in which children 
are recognized as having different learning speeds, and are helped to 
proceed through the various school levels at their ovm unique rate. 

Central to any assessment of school quality under such a theory of education 
is the determination of the correct speed at which each child should progress. 

Scattergrams offer a convenient method of determining the adequacy of 
speed of pupil progress. They are constructed by plotting on a graph the 
juncture of two measures or scores for each pupil. The pattern of the plotted 
scores yields information about the progress of the group, and the location of 
individual plotted points reveals the adequacy of progress of specific students. 
Two types of scattergram plots can be done. The first, based upon the theory 
that mental age is a good predictor of pupil progress, plots mental age against 
academic achievement. The second, based upon the theory that past achievement 
is one of the best indicators of present and future achievement, plots past 
achievement levels against present achievement. Both of these theories have 
been shown in past research to be true as far as educational progress is con¬ 
cerned, but are not necessarily true for prediction of post-graduation Job 
success. In the former situation, understanding of the scattergram is aided 
if mental age is converted into a mental grade score which is comparable in 
scale to an achievement grade equivalent score. A table for conversion of 
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mental age scores to mental grade scores is provided in Appendix A. 

Figure 5 shows a completed plotting of student's grade 5 reading 
comprehension scores with their mental grade scores. Each student's 
mental grade and grade equivalent scores are plotted on the scattergram 
at the junction of their values. Each student may be entered as a tally, 
however, in Figure 5 the tallies have been totaled here for ease of viewing. 
Student A could be one of the six children having a mental grade score of 8 
and an achievement age score of 8, while student B could be the one student 
with a mental grade score of 9 and an achievement age of 12. 

A diagonal line composed of dashes has been drawn through the squares 
which have equal values on both scales. This is an expectancy line since 
children would theoretically be expected to score on achievement tests at 
a level comparable to their mental age. Because no test is a completely 
accurate instrument, the scores cannot be expected to fall exactly along 
the line of expectancy. Errors of measurement would expand the expectancy 
band to at least one grade level above and one grade level below the dashed 
line. This band, within v?hich scores might normally be expected to fall, 
is outlined by two light solid lines, one above and one below the dashed 
line. Tallies found in squares which are cut by, or between, the solid 
lines are to be considered as within expected ranges. Tallies of children 
found above the top solid line suggest the children to be achieving in 
academic skills above that which can be expected in light of their mental 
ages. Those tallies below the lower solid line indicate the children are 
achieving in academic skills at a level lower than could be expected in 
relation to their mental ages. 
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Figure 5 shows many pupils to be achieving below what could be 
expected. Whether the pattern shows achievement levels to be unexpectedly 
high or low, explanations should be sought. First, possible causes outside 
the school should be explored. Such things as changes in the population 
of the community or activities at the public library could produce changes 
in student scores. Failure to account for such factors could lead to 
erroneous conclusions about the effects of school programs. 
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Figure 5. Scattergram of Mental Grade Scores and 
Achievement Grade for Grade 5 Reading 
Comprehension 
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A second source of possible explanations includes the character¬ 
istics of the tests and analytical procedures. If the tests do not well 
represent the curriculum, if there were irregularities in testing procedures, 
or if test results are compiled so as to obscure relationships, the results 
may be irrelevant or biased. 

Finally, factors in the school environment should be scrutinized 
to identify possible explanations for the performance of students. This 
is the ultimate question, for it is the school environment which can be 
changed to improve student achievement. 

The second type of scattergram plots one year's achievement against 
that of a subsequent year. As mentioned before, the theory behind this 
type of evaluation is that previous achievement is related to later achieve¬ 
ment. Of course, schools try to increase the achievement level of their 
charges until they reach their maximum. At the same time, care is taken 
not to push the students to the point of frustration. 

Paradoxically, a tally appearing above or below the expected line can 
be due to either an unusually good score on one axis, or an unusually poor 
score on the other axis. It is the duty of the analyst to determine if 
either of the above situations exists, or if in fact the scores are true 
and valid measures of that student's ability or accomplishment. 

Figure 6 illustrates the plotting of reading scores of third grade 
children against the reading scores of the same children in grade four. 

Once again a diagonal dashed line has been drawn to pass through the 
juncture of expected scores. The solid lines outline the area of probable 


progress. 




Grade Equivaleat Scattergram 


Figure 6. Scattergram of Third Grade Achievement Plotted Against 
Fourth Grade Achievement. 
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The pattern of scores shown by the entries on Figure 6 suggest that the 
fourth grade teacher (if they were year-end tests) placed much emphasis 
upon raising the scores of low achievers. This is revealed by the greater 
number of children appearing above the lower end of the diagonal line. 
Unfortunately, the better achievers may have suffered from a lack of 
attention as shown by the appearance of scores below the line at the upper 
end. Possibly too, the unusual shape could have been due to a change of 
instructional materials that did not adequately challenge the more able 
students. 

The cluster of students above the middle of the line should also 
cause a search for reasons. Maybe, these students lived in an area where 
one of the park supervisors had spent the summer evenings reading stories 
to area children and listening to them read. Or, possibly the children, 
who showed advancement above that expected of them, were involved in an 
experimental reading project. 

Several types of blank scattergrams and a table of scales to be used 
on the margins of grade equivalent scattergrams are provided in Appendix B. 
Percentiles, a common method of reporting pupil scores in high school, may 
also be plotted on a scattergram. A blank percentile scattergram is provided 
for duplication in Appendix B. 

Modes and Clusters 

An added value of scattergrams is that they reveal groupings (modes), 
the spread of scores, and how closely the scores group together. The total 
columns at the side and bottom are similar to that of a line graph. Modes, 
then, can be seen not only through the numbers on the margins but also in 
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the clustering of scores in the body of the tables. In the body of the 
scattergram the greater number of the tallies should cluster closely to the 
dashed line. Groups of students' scores that suggest some forces at work 
may be positive (those above the major grouping) or negative (those below). 
In Figure 5, page 18, mental grade columns 7, 8, and 10 all have distinct 
clusters below the line. The column for mental grade 4 has a mode above 
the line. Two students are distinctly superior in achievement for achieve¬ 
ment age 12. Why are the scores of these students located where they are? 
Are the mental ages improperly measured for any of the mavericks? Are the 
10 students in mental grade column 8, rows 5 and 6, similar in some way? 

Are they also similar to the six and five students in columns 7 and 6? Are 
some of the eight students in column 5 also like the other groups of below 
achievers just discussed. 

All of these children should be identified by name. Then similar¬ 
ities among them could be searched out. The children in lower modes might 
have been those seated in the back of the cafeteria in the illustration 
given earlier, they might have missed school because of a flu epidemic, or 
their classroom seats may be near a poorly adjusted child. Too, any sag 
at the top of the distribution should immediately warn the observer that 
the test used may not have enough questions which cover advanced curricular 
materials. Such a test has a low ceiling and able students "top out" the 
test. A flat bottom at the lower end of the distribution may sugge-st a 
false bottom to the test. In the latter situation, correctly answering 
only one question can yield a score well above the actual ability level 


of the child. 
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Scattergraras, as can be seen, have many uses. They can describe status at 
some point in time (Figure 5) or progress over a period of time (Figure 6). 

They are easy to construct, allow observation of two factors at the same 
time, while pointing to trouble and/or strong clusters and modes of students 
which deserve analytic attention. Through the use of color coded tallies, 
different groups of students may be followed in order to determine experi¬ 
mental, program, and/or administrative effects. Effort taken to construct 
scattergrams is minor in relation to their value. 

Tree Diagram 

The question, "How good is my school?", can also be answered through 
use of the tree diagram. The procedure involves following the progress of 
groups of students through school, determining their success at several 
points. The process may be done historically, that is, using past records 
to determine adequacy of present achievement, or it can be done concurrently 
to determine progressive changes. 

The basic assumption behind tree diagrams is that a good school is 
one whichkeeps its children achieving as well as they have ever done, if 
not better. Accordingly, a child, who has done above average scholastic 
work in the elementary school, should continue to do good work in junior 
and senior high. The measure of quality is the percent of students who 
continue to do at least the same level work throughout their school life, 
as opposed to those who do not. Districts having high turnover will find 
this analytic method of limited use, as will those districts with poor 


record files. 
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The tree diagrams are constructed by first dividing the long-term 
students into three groups according to their scores. Starting with the 
elementary students, their achievement is judged to have been "Above 
Average," "Average," or "Below Average." Table 2 is provided to help the 
analyst determine the rating of single scores. 


Table 2 
Rating Scales 


Rating 

Coimnercial Test Scores 

Teacher Grades I 

Percentile 

Difference from Average 
Grade Equivalent 

Percentage 

Letters 

Above 

Average 

85-99 

Two grades above 
average and up 

85-100 

A,B 

E,G 

Average 

40-84 

Average up to 2 grades 
above 

75-84 

C 

S 

Below 

Average 

1-39 

Up to average 

1-74 

D,E,F 

U 


Those students whose individual ratings were better than average throughout 
the elementary years would be assigned an "Above Average" rating. More than 
two scores or grades below average is cause for a "Below Average" rating. 
Students remaining, those who earned average grades, are assigned an "Average" 
rating. The number of "Above Average," "Average," or "Below Average" students 
is then noted on the tree. Percentages of the three groups are then computed 
and entered on the tree diagram (Figure 7). The reader will note that 


the tree lies on its side. 
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Junior high achievement for students in each of the three elementary 
groups--Above Average, Average, Below Average--is judged in the same manner 
and entered with percentages on the tree diagram (see Figure 7). Hopefully, 
all students judged to be "Above Average" students in the elementary grades 
will receive the same rating in the junior high school years. Experience 
has shown such is seldom the case, though "Above Average" elementary students 
are rarely judged to be "Below Average" in junior high. "Average" students 
can and do go to both "Above Average" and "Below Average" ratings with the 
bulk staying "Average." Unfortunately, many "Below Average" students have 
been found to remain "Below Average." 

Each student in the nine junior high groups are then rated according 
to their senior high grades or scores. Totals are found for each of the 
27 groups. Totals may be entered on the tree diagram and percentages 
secured for each as shown on Figure 7. 

As can be readily seen, the tree diagram helps one follow the progress 
of a particular group of students. Of the 32 children judged to be "Above 
Average" in the elementary school, 28 continued with that ranking in the 
junior high school. However, four dropped to lower rankings in the junior 
high years. Of the two students who dropped from "Above Average" in the 
elementary grade to a "Below Average" ranking in the junior high, one 
stayed in the "Below Average" category in the senior,high, while one recovered 
his "Above Average" status. 

Many of the "Average" rated elementary children were induced to 
greater achievement in the junior high and likewise into the senior high 
school (see Figure 7). That nine of the "Average" group finally ended as 
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Figure 7. Tree Diagram 
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"Below Average" should be cause for concern. Of the 28 "Below Average" 
achieving students, 24 remained in that grouping throughout school. Three 
of the 28 students who were "Below Average" in the elementary years ended 
as "Average" students in their high school years. 

Because of the gross method of separating the pupils when using the 
tree diagram method, some children who are on the borders of the groups 
can be expected to switch from one group to another. That does not mean 
that changes should be disregarded. Each drop should be cause for concern 
and a key to where to start looking for reasons why the drop occurred, 

One of the questions answered by tree diagram analysis is, "How 
well have the children progressed in school?" This same question can be 
answered by totaling the number of children found in each of the three 
rated categories for elementary, junior, and senior high levels. Table 3 
has been constructed from the numbers of students shown in Figure 7. 


Table 3 

Totaling of Tree Diagram 




■SJEBSSlBsm 


Rating 

N 

% 

N 

7o 

N 

7 

/o 

Above Average 

32 

23.9 

40 

29.9 

40 

29.9 

Average 

74 

55.2 

58 

43.3 

59 

44.0 

Below Average 

28 

20.9 

36 

26.9 

35 

26.1 

Totals 

134 

100 

134 

100.1 

134 

100 
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Comparison of the percentage figures among the various levels shows 
that this school district was able to increase the number of students in 
the "Above Average" category, but unfortunately found the percent of stu¬ 
dents in the "Below Average" category to increase, also. The greatest 
change in numbers, and therefore percents, occurred between the elementary 
and junior high levels. Though this large change could have been due to 
score shifts of those pupils near the boundaries of the categories, it 
could be due to other reasons as well. Different teacher grading policies, 
dissimilar tests used in the two levels, poor curricular coordination, 
and/or overcrowding could have been a few of the reasons. In any event 
the illustrative data tends to answer the question with a statement that 
the "Average" student is not progressing as well as one could wish. 

Tree diagrams can be constructed similar to Figure 7 using teacher 
grades or commercial achievement tests. The choice of measure would depend 
upon completeness of data, or trust one has in the accuracy of the measure. 
However, enough data is usually available in a district's files to allow 
analysis by tree diagraming using one type. 

Blank data collection charts, and blank summary tables and cards can 
be found in Appendix B and may be reproduced. They may be used in those 
situations where data and information are recorded on several source docu¬ 
ments. The data card is especially helpful in pulling several pieces of 
information together on a single source. Further, the cards lend themselves 
to quick sorting into the three categories illustrated in Figure 7. Sorting 
can also be done by electronic data processing equipment. 
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Tree diagrams can also be used to answer questions dealing with sub¬ 
classifications of students, for example, tree diagrams can be used to 
Study effects of integration upon the achievement of black or white students. 
In this situation a historical study could first be made of the progress of 
both categories of students previous to integration. Measures could be 
drawn from school records of first, third and sixth grade teacher grades or 
commercial achievement tests. After integration the same type of data, 
recorded at similar grade levels, can be placed upon tree diagrams. Up to 
seven years could pass before the analysis of the effect of integration upon 
the scores of elementary school pupils could be completed, though interim 
analyses could be made along the way. Judgment of the effect of integration 
would depend upon comparison of pre-integration and post-integration percen¬ 
tage figures at each of the three grade levels. A positive effect of inte¬ 
gration would be shown by a decrease in the number and percent of pupils in 
"Below Average” categories and an increase in "Average" and "Above Average" 
categories. 

Analyses can also be done using sex, I.Q., parental education, 
parental occupation, or other classifications. To do so, separate tree 
diagrams are made for each division of the classification and comparison 
is made as illustrated for the integration study. Space is provided on 
the forms in Appendix B for classifications other than those mentioned. 

Concurrent tree diagram analysis is accomplished by filling in the 
branches of the tree whenever data becomes available- The question changes 
from, "How have the children progressed?" to "How are the children pro¬ 
gressing?" This type of evaluation is helpful in following children who 
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have participated in special programs. Their progress can be compared with 
those not included. Three years is the sensible limit for tree diagram 
analysis. Above three years, the data must be grouped and averaged by a 
larger segment of time, as in Figure 7. 

Modules 

For the most part the types of analyses given in the previous 
portions of this document are generally useful to those districts having 
their students in traditional graded organizations. Indeed, one of the 
advantages of the graded system is the tried and true measurement methods 
and analytical techniques available as aids to teachers and administrators. 
However, such is not the situation for districts using individualized 
instructional methods. One of the sharpest criticisms leveled at schools, 
which are organized so that students may advance at their own rate, is that 
they cannot answer the question, "How are the children progressing?" 

Involved in this question is consideration of proper speed of progress. 

The traditional graded system of organization bases evaluation upon 
the progress of the group, A graded group is usually composed of children 
of the same age. Children are scored according to their position relative 
to the group, A fast learner is rated as A, excellent, or 90, while the 
slower child, even though he works hard, earns D, poor, unsatisfactory, or 
65. Under this management system some children will always be "Above Average" 
and some will always be "Below Average," Obviously, an evaluation system 
is needed which adequately considers the unique ability of the child, when 
assignments are passed out and marks are given. 
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Expression of pupil progress in terms of modules, or units of 
curriculum, offers evaluation possibilities for schools organized on the 
individualized instruction system, whereas, scattergrams and tree diagrams 
offer evaluation methods for schools organized on the graded system. 

Monitoring Pupil Progress 

Modules are single units of study in a curricular sequence. If 
children are allowed to progress at their own rate in harmony with their 
own potential, individuals in any group of children will be working on 
many different modules. The logistics of merely keeping track of the 
progress of children becomes difficult. A simple and sensible solution is 
to divide the curriculum into short segments--modules--and then to construct 
a chart which has modules sequenced across the top and children's names 
listed down the side. Then, each time a child completes a module a check 
mark can be entered under the correct module for that child. Table 4 shows 
how this might be done for a single age or grade grouping. It could repre¬ 
sent a classroom after two months of the school year have passed. Students 
4 and 10 are slow learners, having fallen behind the main body of their 
fellows. Some are faster learners (number 7, 19 and 25) having pulled ahead. 

Estimating Pupil Progress 

Though Table 4 gives the teacher and administrator a satisfactory re¬ 
port of individual pupil status, it does not show whether the pupil's position 
is good or poor. Some estimate of the potential of each pupil is necessary. 
The simplest individual estimate can be made from a child's mental age or 
I.Q. Of course, each estimate should be modified by teacher judgment and/or 
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review of each student's record file. Estimates of individual pupil 
expected progress can be entered on a chart. Such charts aid the teacher 
in planning the curriculum for the coming year. Table 5 has been constructed 
to show the modules which children of a classroom might be expected to com¬ 
plete during the school year. The X's represent expected mastery. For 
example, students 4 and 10 are projected to progress at a slower pace and 
will not complete as many modules as other students. On the other hand, 
students 2, 7, 14, 19 and 25 are expected to complete more modules than the 
bulk of students. Student 16, who has fallen behind (possible due to illness), 
is expected to progress a normal ten modules during the year. 

Appendix C contains illustrative charts of mastery for children of 
differing mental ages or I.Q. levels. In these tables the mastery of modules 
is seen to increase as the mental age or I.Q. rises. The tables in Appendix C 
are illustrative of a modular system of 10 equal units each year, though the 
decimals may also represent parts of the whole module that should be earned. 
Individualized instructional units set up by districts need not have regu¬ 
larly spaced modules during the year, nor contain the same number of modules 
each year. 

Mastery of Expectancy 

Neither of the above tables are, alone, sufficient for judging the 
adequacy of progress of individual pupils. The final step in converting the 
previous progress module and expectancy tables into an evaluation tool is to 
merge the two. Table 6 is such a tool. It combines expectancies as X's and 
actual pupil progress as circles around the X's. Pupil progress is judged 
by determining how far the children have advanced toward completion of the 
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pro jected year’s work. Generally, the students whose progress has been 
charted on Table 6, have advanced the two modules as expected. Those 
students who learn more slowly have advanced only one module. Faster 
learners have advanced more than two modules, with the exception of 
student 7. The reason student 7 is slowing down should be sought. Is it 
because of absence due to illness? Could family problems be interfering 
with school study? In contrast students 14 and 25 have exceeded what 
might have been expected. Have their projected achievements been misjudged? 
Is student 14 a late bloomer? These and other questions flow naturally 
from scanning the table. 

Evaluation of total school quality is readily determined by summar¬ 
izing each teacher's tables on a form such as Table 7. Few students should 
be found in either the "Above Expected" and "Below Expected" categories. 
Those that are found in either should be immediately studied. Study of 
"Above Expected" children may yield clues to program, curricular content, 
or methodologies that can be used to further the learning of other children. 
It may also reveal unhealthy pressures for higher achievement. Of course, 
those found "Below Expected" should be cause for concern. 

Expectancy progress tables therefore perform a four-fold function. 
First, they are a control device whereby teachers are kept continually 
aware of student abilities and progress. Second, they are alarms which 
signal unusual academic behavior. Third, they are evaluation devices 
which provide to both teachers and administrators a method of measuring 
their effectiveness. Finally, use of expectancy progress tables helps 
answer the question, "How are my children progressing?" 
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Recap 

Several methods of measuring school effectiveness have been offered. 
Most involve the use of simple arithmetic. Often numbers are placed in 
chart, graphic or tabular form. Analysis is highly visual. Because the 
procedures to be followed are simple, they are easily done. Since the 
method of presentation is for-the-most-part visual, the results of the 
evaluation are readily understood. The evaluation techniques presented 
are thus useful to those not interested in a robust, mathematically 
oriented, statistical analysis. Their strength and their weakness lie in 
their simplicity. 
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APPENDIX A 

MENTAL AGE TO MENTAL GRADE CONVERSION TABLE 

Derivation of Mental grade scores; 

Mental ages are usually expressed in months. /M ental age is 
the dividend of the I.Q. formula: mental age -i chronological age = 

I.Q. /. To convert the figure to years, it is divided by 12. Then 

so it will correspond to the grade for which it normally compares, 

5 (five years from birth to beginning of first grade) is subtracted 
from. it. Thus, a mental age of 107, divided by 12 and minus 5 equals 
a mental grade of 3.9 
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Directions: Find the mental age figure (expressed as months) in body of 

table. Go straight left to the left margin to determine the year, then go 
straight up from the monthly mental age to the upper row for the decimal 
fraction. For example, a monthly mental age of 128 yields 5th grade on 
the left and .6 months on the top. 
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APPENDIX B 

SGATTER.GRAM SCALE TABLES AND BLANK FORMS 
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GRADE EQUIVALENT SCALES FOR SCATTERGRAM 
ROWS* AND COLUMNS'^'-' 


Grade 

Cell 

1 

2 

3 

4 

5 

6 

7 

8 

_ 

9 

10 

1 

.4 

.6 

.8 

1.0 

1.2 

1.4 

1.6 

1.8 

2.0 

2.2 

2 

.8 




2.4 

2.8 

3,2 

3.6 

4.0 

4.4 

■ 

3 

1.2 

1.8 

2.4 


3.6 

4.2 

4,8 

5.4 

6.0 

6.6 

4 

1.6 

2.4 

3.2 



5.6 

jm 

7.2 

8.0 

8.8 

5 

2.0 

3.0 



6.0 

7.0 

8.0 




6 

2.4 

3.6 

4-8 

6.0 

m 

8.4 





7 

2.8 

4,2 

5,6 

7.0 

8.4 

9,8 




15.4 

8 

3.2 

4.8 


8.0 

9.6 

11.2 




17.6 

9 

3.6 

m 

7.2 

9.0 

_ 

10.8 





19.8 


*Row headiags are always for the higher of the two grades with the 
scale on the left running from low values at the bottom to 
greater values at the top of the scattergram. 


**CoIurati headings are always for the lower of the two grades with 
the scale on the top of the scattergram running from left as 
smaller numbers to the right in increasingly greater values. 

- Scattergrams need not cover closely sequenced grades. Grade 3 
versus grade 6, grade 4 versus grade 6, and grade 6 versus 
grade 9 are also viable uses. 
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Scattergram of Mental Grade Scores versus Achievement Grade 
Scores. (Place scales on left and top so same figure 
junctures on dashed line.) 
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Grade Equivalent Scattergram 

(Place scales on left and top so the grade which is the basis for each 
scale junctures in the square. See page 20.) 
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Percentile Scattergram 
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Cont alns: 


APPENDIX G 

FORMS FOR TREE DIAGRAM ANALYSIS 


Collection Sheets, Synthesis Cards and Forms, 
and Several Tree Diagram Forms 
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Achievement Tests, Elementary Level 
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Achievemsnt T6sts, Junior and Sanior High Levels 
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Teacher Grades, Elementary Level 
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Intelligence Test Record 


































Summary Sorting Card 
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Sunmary Sheet 
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Units plus Modules per Grade 
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Expectancies by I.Q. and Grade 
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Expectancies by Mental Age and Grade 










































Blank Module Mastery Chart 





